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SUBJECT:  Quality  Control  Review  of  the  PricewaterhouseCoopers  LLP  FY  2014  Single  Audit 
of  Carnegie  Mellon  University  (Report  No.  DODIG-2016-034) 

We  are  providing  this  report  for  your  information  and  use.  We  considered  management 
comments  on  a  draft  of  this  report  when  preparing  the  final  report.  Comments  from 
PricewaterhouseCoopers  LLP  were  responsive  to  our  recommendations;  therefore,  we  do 
not  require  additional  comments. 

As  the  cognizant  Federal  agency  for  Carnegie  Mellon  University,  we  performed  a 
review  of  the  PricewaterhouseCoopers  LLP  single  audit  report  and  supporting  audit 
documentation  for  the  year  ended  June  30,  2014.  The  purpose  of  our  review  was  to 
determine  whether  the  single  audit  was  conducted  in  accordance  with  auditing  standards 
and  the  requirements  of  Office  of  Management  and  Budget  (OMB)  Circular  A-133,  "Audits 
of  States,  Local  Governments,  and  Non-Profit  Organizations.”  We  conducted  this  quality 
control  review  in  accordance  with  the  "Quality  Standards  for  Inspection  and  Evaluation" 
published  by  the  Council  of  the  Inspectors  General  on  Integrity  and  Efficiency  (CIGIE)  in 
January  2012. 

Carnegie  Mellon  University  complied  with  OMB  Circular  A-133  reporting  requirements 
for  the  FY  2014  single  audit.  However,  PricewaterhouseCoopers  LLP  did  not  fully  comply 
with  auditing  standards  and  OMB  Circular  A-133  requirements  for  the  Carnegie  Mellon 
University  FY  2014  single  audit.  Specifically,  the  auditors  did  not  adequately  perform 
and  document  audit  procedures  for  the  review  of  the  cash  management  requirement. 
PricewaterhouseCoopers  LLP  needs  to  perform  additional  audit  work  for  this  direct  and 
material  requirement.  PricewaterhouseCoopers  LLP  agreed  to  take  corrective  action  in 
response  to  our  recommendations. 

We  appreciate  the  courtesies  extended  to  our  staff.  For  additional  information  on  this 
report,  please  contact  Ms.  Carolyn  R.  Hantz  at  (703)  604-8877  (DSN  664-8877). 


Deputy  Inspector  General 
Policy  and  Oversight 
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Objective 

As  the  cognizant  Federal  agency1  for  the  Carnegie  Mellon  University  (Carnegie  Mellon), 
we  performed  a  quality  control  review  of  the  PricewaterhouseCoopers  LLP  (PwC) 
single  audit  report  and  supporting  audit  documentation  for  the  audit  period  of 
July  1,  2013,  through  June  30,  2014.  Our  objective  was  to  determine  whether  the 
single  audit  was  conducted  in  accordance  with  generally  accepted  government 
auditing  standards,  generally  accepted  auditing  standards,  and  the  requirements 
of  Office  of  Management  and  Budget  (OMB)  Circular  A-133,  "Audits  of  States,  Local 
Governments,  and  Non-Profit  Organizations."  Appendix  A  contains  additional 
details  on  our  scope  and  methodology  and  identifies  prior  quality  control  reviews. 
Appendix  B  lists  the  compliance  requirements  that  PwC  determined  to  be  applicable 
to  the  FY  2014  audit. 

Background 

Carnegie  Mellon  University 

Carnegie  Mellon  is  a  private,  not-for-profit  educational  and  research  institution. 
Carnegie  Mellon  enrolls  about  13,000  students  and  grants  approximately 
4,100  bachelor’s  degrees,  master’s  degrees  and  doctoral  degrees  each  year. 

A  substantial  portion  of  Carnegie  Mellon’s  revenues  are  from  sponsored  research 
and  other  projects  under  Federal,  state,  industrial,  and  other  contracts.  During 
FY  2014,  Carnegie  Mellon  expended  $379.9  million  in  Federal  funds.  Of  the 
$379.9  million,  Carnegie  Mellon  expended  $322.9  million  on  one  major  program, 
the  research  and  development  cluster;  including  $187.2  million  expended  on 
Department  of  Defense  awards.  Carnegie  Mellon  engaged  PwC  to  perform  the 
FY  2014  single  audit. 

PricewaterhouseCoopers  LLP 

PwC,  a  member  firm  of  PricewaterhouseCoopers  International  Limited,  provides 
a  wide  array  of  business  services,  including  audit  and  assurance,  business  and 
Government  consulting,  and  tax  preparation  and  planning.  PwC  maintains  its  own 
system  of  internal  quality  control  over  its  accounting  and  auditing  practices  as 
required  by  the  American  Institute  of  Certified  Public  Accountants.  PwC’s  office 
in  McLean,  Virginia,  performed  Carnegie  Mellon’s  FY  2014  single  audit. 


1  Office  of  Management  and  Budget  Circular  A-133  states  that  the  cognizant  agency  is  the  Federal  agency  that  provides  the 
predominant  amount  of  direct  funding  to  a  non-Federal  entity  and  is  the  Federal  agency  designated  to  perform  quality 
control  reviews. 
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Single  Audit 

Public  Law  98-502,  "The  Single  Audit  Act  of  1984,"  (the  Act)  as  amended,  was 
enacted  to  promote  sound  financial  management  of  Federal  awards  administered 
by  non-Federal  entities  and  to  establish  a  uniform  set  of  auditing  and  reporting 
requirements  for  all  Federal  award  recipients  that  are  required  to  obtain  a  single 
audit.  OMB  Circular  A-133  establishes  policies  that  guide  the  implementation  of  the 
Act  and  provides  an  administrative  foundation  for  uniform  audit  requirements  of 
non-Federal  entities  administering  Federal  awards.  Entities  that  expend  Federal 
funds  of  $500,000  or  more  in  a  year  are  subject  to  the  Act  and  OMB  Circular  A-133 
requirements.  Therefore,  they  must  have  an  annual  single  or  program-specific 
audit  performed  in  accordance  with  generally  accepted  government  auditing 
standards  and  submit  a  complete  reporting  package  to  the  Federal  Audit 
Clearinghouse.  The  single  audit  includes  an  audit  of  the  non-Federal  entity’s 
financial  statements  and  Federal  awards  as  described  in  OMB  Circular  A-133. 

Review  Results 

Carnegie  Mellon  complied  with  OMB  Circular  A-133  reporting  requirements. 

However,  PwC  did  not  fully  comply  with  auditing  standards  and  OMB  Circular  A-133 
requirements  for  the  Carnegie  Mellon  FY  2014  single  audit.  Specifically,  the 
auditors  did  not  adequately  perform  and  document  audit  procedures  for  their 
review  of  the  cash  management  requirement.  As  a  result,  PwC  needs  to  complete 
additional  audit  work  to  support  its  audit  conclusions  and  the  overall  opinion  on 
compliance  with  requirements  for  the  research  and  development  cluster. 

Management  Comments  and  DoDIG  Response 

PricewaterhouseCoopers  LLP  agreed  to  take  corrective  action  in  response  to  our 
recommendations.  Management  comments  were  responsive  and  no  additional 
comments  are  needed.  PwC  agreed  to  perform  additional  testing  and  provided 
additional  information  to  support  the  conclusions  on  the  cash  management 
requirement.  We  will  review  all  information  provided  and  additional  testing 
performed  during  our  follow-up  procedures  to  ensure  actions  fully  address 
deficiencies  cited  in  this  report.  Management  comments  are  included  in  their 
entirety  at  the  end  of  this  report.  Appendix  C  includes  a  summary  of  management’s 
comments  on  our  finding  along  with  our  response. 
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Finding 

Audit  Procedures  and  Documentation  of  the 
Cash  Management  Compliance  Requirement 

PwC  did  not  adequately  perform  and  document  audit  procedures  to  support 
conclusions  on  Carnegie  Mellon’s  internal  control  over  and  compliance  with  cash 
management  requirements.  PwC  identified  the  cash  management  compliance 
requirement  as  direct  and  material  to  the  research  and  development  cluster. 
However,  the  audit  procedures  documented  did  not  achieve  the  audit  objective  on 
the  cash  management  compliance  requirement  for  the  research  and  development 
cluster.  In  addition,  the  audit  documentation  contained  errors  and  inaccuracies. 

As  a  result  of  these  deficiencies,  the  audit  documentation  did  not  provide  sufficient 
appropriate  evidence  to  support  audit  conclusions  on  cash  management.  PwC  must 
document  and  perform  additional  audit  procedures  before  Federal  agencies  can  rely 
on  the  overall  opinion  on  Carnegie  Mellon’s  compliance  with  requirements  for  the 
research  and  development  cluster. 


Audit  Procedures 

PwC’s  documented  audit  procedures  for  testing  the  cash  management 
compliance  requirement  do  not  adequately  address  the  audit  objective  identified 
in  the  2014  OMB  Circular  A-133  Compliance  Supplement.  PwC  documented  that 
Carnegie  Mellon  receives  funds  on  a  reimbursement  basis.  The  OMB  Circular  A-133 
Compliance  Supplement  states  that  when  funds  are  received  on  a  reimbursement 
basis,  the  audit  objective  is  to  ensure  program  costs  are  paid  with  entity  funds 
before  requesting  reimbursement  from  the  Federal  Government.  PwC’s  documented 
audit  procedures  indicated  that  PwC  verified  that  the  reimbursement  request  did 
not  exceed  cumulative  expenses  and  that  payments  received  were  equal  to  the 
amount  requested.  PwC  also  performed  procedures  to  verify  Carnegie  Mellon’s 
approval  of  the  reimbursement  request.  The  audit  procedures,  as  documented,  do 
not  provide  sufficient  evidence  to  conclude  on  Carnegie  Mellon’s  compliance  with 
the  objective  of  the  cash  management  requirement. 

Compliance  Testing 

PwC  selected  a  sample  of  reimbursement  requests  and  performed  two  audit 
procedures  to  test  Carnegie  Mellon’s  compliance  with  the  cash  management 
requirement.  PwC  verified  that  the  amount  requested  for  reimbursement  did  not 
exceed  total  expenditures  incurred  for  the  award.  The  audit  documentation  was 
not  clear  on  whether  the  auditor  used  cumulative  incurred  expenses  at  year  end 
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or  expenses  incurred  at  the  time  of  the  reimbursement  request.  In  either  case, 
this  audit  procedure  did  not  provide  evidence  that  met  the  audit  objective  for  the 
requirement  because  PwC  is  relying  on  the  expense  accounts2  without  documenting 
evidence  of  Carnegie  Mellon’s  payment  of  the  expenses.  PwC  also  verified  that  the 
amount  received  matched  the  amount  requested;  however,  this  procedure  also  did 
not  achieve  the  audit  objective  for  the  requirement. 

We  asked  PwC  to  explain  why  the  documentation  did  not  include  evidence  of  the 
date  Carnegie  Mellon  paid  expenses.  PwC  informed  us  that  its  reliance  on  expense 
accounts  is  based  on  the  understanding  that  Carnegie  Mellon  issues  monthly 
reimbursement  requests  for  the  previous  month's  expenses  and  an  analysis  of 
the  annual  accounts  payable  turnover  ratio.  The  turnover  ratio  used  is  a  broad 
assessment  of  the  annualized  average  time  it  takes  Carnegie  Mellon  to  pay  the 
expenses,  but  does  not  provide  proper  evidence  of  actual  payment  before  the 
request  for  reimbursement.  Additionally,  PwC  did  not  document  audit  procedures 
to  validate  its  understanding  of  Carnegie  Mellon’s  processes.  As  a  result,  the  audit 
documentation  did  not  include  sufficient  evidence  that  the  auditors  met  the  audit 
objective  to  conclude  on  compliance  with  the  cash  management  requirement. 

Testing  of  Internal  Controls  over  Compliance 

PwC  performed  two  audit  procedures  for  testing  internal  controls.  PwC  reviewed 
Carnegie  Mellon’s  approval  of  the  reimbursement  request  and  Carnegie  Mellon’s 
letter  of  credit  quarterly  reconciliations.  The  audit  documentation  did  not  provide 
clear  evidence  of  how  the  individual  approving  the  reimbursement  ensured 
compliance  with  the  cash  management  requirement.  The  test  of  quarterly 
reconciliations  provides  limited  coverage  for  the  overall  cash  management 
requirement  because  the  letter  of  credit  is  applicable  to  only  5.7  percent  of  FY  2014 
Federal  expenditures.  As  a  result,  we  determined  that  the  audit  documentation  did 
not  provide  sufficient  evidence  to  support  the  audit  conclusions  on  internal  control 
and  to  plan  the  compliance  testing  for  the  cash  management  requirement. 

Audit  Documentation 

The  audit  documentation  of  internal  control  and  compliance  testing  for  the  cash 
management  requirement  contained  numerous  errors  and  inaccuracies.  Auditing 
standards  require  that  audit  documentation  be  in  sufficient  detail  to  enable  an 
experienced  auditor  with  no  previous  connection  to  the  audit  to  understand  the 
work  performed,  the  evidence  obtained,  and  the  conclusions  reached  to  meet  the 
audit  objectives. 


2  Under  the  accrual  basis  of  accounting,  Carnegie  Mellon's  expenses  were  accounted  for  when  incurred,  regardless  of 
whether  they  have  been  paid  or  will  be  paid. 
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We  noted  that  the  auditor's  spreadsheet  documenting  the  internal  control  and 
compliance  testing  identified  reimbursement  request  dates  that  were  for  a  prior 
fiscal  year  and  payment  received  dates  that  were  before  the  reimbursement  request 
date.  Due  to  the  discrepancies  identified,  we  decided  to  perform  retesting  and 
selected  6  of  the  25  sample  items  for  review.  We  were  able  to  determine  that  the 
dates  on  the  spreadsheet  were  entered  incorrectly  for  five  of  the  six  sample  items. 
However,  Carnegie  Mellon  informed  us  that  the  sixth  item  was  for  a  fixed-price 
award,  which  would  not  be  subject  to  OMB  Circular  A-133  audit  requirements. 

When  asked,  PwC  explained  that  the  sample  item  described  on  the  spreadsheet 
was  not  the  reimbursement  request  actually  tested.  However,  in  its  response  to 
a  draft  of  this  report,  PwC  agreed  that  the  sample  item  tested  was  the  fixed-price 
award.  During  our  retesting,  we  noted  errors  in  the  audit  documentation  including 
the  identification  of  a  different  approver  for  the  reimbursement  request  than 
the  one  identified  on  the  testing  spreadsheet  and  errors  in  dates.  Due  to  these 
inaccuracies,  we  are  unable  to  rely  on  the  audit  documentation  for  the  cash 
management  requirement. 

PricewaterhouseCoopers  LLP  Comments  on  the  Finding 
and  Our  Response 

A  summary  of  PwC’s  comments  on  the  finding  and  our  response  is  in  Appendix  C. 
See  the  Management  Comments  section  for  the  full  text. 

Recommendations,  Management  Comments, 
and  Our  Response 

Recommendations 

We  recommend  that  the  Audit  Partner,  PricewaterhouseCoopers  LLP: 

1.  Document  and  perform,  as  necessary,  sufficient  audit  procedures  for  the 
FY  2014  single  audit,  at  no  additional  cost  to  the  Government,  to  support 
conclusions  on  Carnegie  Mellon  University's  internal  control  over  and 
compliance  with  cash  management  requirements. 

2.  Provide  the  Department  of  Defense  Office  of  Inspector  General  with 
the  audit  documentation  on  the  FY  2014  audit  that  demonstrates 
corrective  actions  taken  to  address  the  reported  deficiencies  for  the 
cash  management  requirement. 
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PricewaterhouseCoopers  LLP  Comments 

PricewaterhouseCoopers  LLP  agreed  to  take  corrective  action  and  stated  that 
PricewaterhouseCoopers  LLP  will  perform  additional  cash  management  tests  and 
correct  the  errors  found  in  the  testing  spreadsheet.  PricewaterhouseCoopers  LLP 
also  provided  additional  information  on  the  audit  documentation  to  support  the 
testing  on  the  cash  management  requirement. 

Our  Response 

PricewaterhouseCoopers  LLP  comments  are  responsive  to  our  recommendations. 
No  additional  comments  are  required.  We  will  verify  all  information  provided 
and  actions  taken  during  our  follow-up  procedures  to  ensure  actions  are  sufficient 
to  address  deficiencies  identified.  Management  comments  are  included  in  their 
entirety  at  the  end  of  this  report. 
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Appendix  A 

Scope  and  Methodology 

We  reviewed  the  PwC  FY  2014  single  audit  of  Carnegie  Mellon  submitted  to  the 
Federal  Audit  Clearinghouse  on  January  16,  2015,  using  the  2015  edition  of  the 
Council  of  the  Inspectors  General  on  Integrity  and  Efficiency  (CIGIE)  "Guide  for 
Quality  Control  Reviews  of  OMB  Circular  A-133  Audits."  CIGIE  approved  this  guide 
for  performing  quality  control  reviews  of  single  audits.  We  conducted  our  review 
from  April  2015  through  September  2015  in  accordance  with  the  CIGIE  "Quality 
Standards  for  Inspection  and  Evaluation."  We  did  not  review  the  audit  procedures 
performed  related  to  the  student  financial  assistance  cluster.3  Our  review  focused 
on  the  following  qualitative  aspects  of  the  single  audit: 

•  qualification  of  auditors, 

•  independence, 

•  due  professional  care, 

•  planning  and  supervision, 

•  audit  follow-up, 

•  internal  control  and  compliance  testing  for  the  research  and 
development  cluster, 

•  schedule  of  expenditures  of  federal  awards,  and 

•  data  collection  form. 

Use  of  Computer-Processed  Data 

We  did  not  use  computer-processed  data  to  perform  this  review. 

Prior  Coverage 

Since  October  1,  2010,  we  have  performed  five  quality  control  reviews  related  to 
PwC's  OMB  Circular  A-133  audits.  Unrestricted  DoD  IG  reports  can  be  accessed  at 

http://www.dodig.mil/pubs/index.cfm. 

•  DoD  IG  Report  No.  DODIG-2015-112,  “Quality  Control  Review  of  the 
PricewaterhouseCoopers  LLP  and  the  Defense  Contract  Audit  Agency 
FY  2013  Single  Audit  of  the  MITRE  Corporation/'  April  30,  2015 


3  The  Department  of  Defense  did  not  provide  any  funds  in  the  student  financial  assistance  cluster.  The  Department  of 
Education  was  the  only  Federal  agency  to  provide  funds  to  this  cluster  and  has  the  expertise  on  the  requirements  for 
this  major  program.  The  Department  of  Education  did  not  elect  to  participate  in  this  quality  control  review. 
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•  DoD  IG  Report  No.  DODIG-2014-078,  "Quality  Control  Review  of  the 
PricewaterhouseCoopers,  LLP  FY  2012  Single  Audit  of  Massachusetts 
Institute  of  Technology,”  June  5,  2014 

•  DoD  IG  Report  No.  DODIG-2013-125,  "Quality  Control  Review  of  the 
PricewaterhouseCoopers,  LLP  FY  2011  Single  Audit  of  SRI  International," 
August  29,  2013 

•  DoD  IG  Report  No.  DODIG-2013-048,  "Quality  Control  Review  of 
the  PricewaterhouseCoopers,  LLP  and  the  Defense  Contract  Audit 
Agency  FY  2010  Single  Audit  of  the  Institute  for  Defense  Analyses," 
February  20,  2013 

•  DoD  IG  Report  No.  D-2011-6-004,  "Report  on  Quality  Control  Review 
of  the  PricewaterhouseCoopers,  LLP  and  the  Defense  Contract  Audit 
Agency  FY  2008  Single  Audit  of  the  Charles  Stark  Draper  Laboratory, 
Incorporated,"  February  28,  2011 


8  |  DODIG-2016-034 


Appendixes 


Appendix  B 

Compliance  Requirements 


Table.  PwC’s  Determination  of  the  Applicability  of  Compliance  Requirements  for  the 
Research  and  Development  Cluster 


OMB  Circular  A-133  Compliance  Requirements 

Applicable 

Not  Applicable/ 
Not  Material 

Activities  Allowed/Unallowed 

X 

Allowable  Cost/Cost  Principles 

X 

Cash  Management 

X 

Davis-Bacon  Act 

X 

Eligibility 

X 

Equipment  and  Real  Property  Management 

X 

Matching,  Level  of  Effort,  Earmarking 

X 

Period  of  Availability  of  Federal  Funds 

X 

Procurement,  Suspension,  and  Debarment 

X 

Program  Income 

X 

Real  Property  Acquisition  and  Relocation  Assistance 

X 

Reporting 

X 

Subrecipient  Monitoring 

X 

Special  Tests  and  Provisions 

X 
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Appendix  C 

PricewaterhouseCoopers  LLP  Comments  on  the  Finding 
and  Our  Response 

PwC  Comments  on  Compliance  Testing  (first  paragraph) 

PwC  stated  that  its  objective  was  clearly  designed  to  determine  whether  the  invoice 
amount  agrees  with  or  is  less  than  the  total  expenditures  "at  period-end."  PwC 
also  stated  that  PwC  auditors  did  not  use  the  "Total  Expenditures"  column  included 
in  the  spreadsheet  for  their  testing  and  that  the  expenditures  at  period-end 
represented  the  billing  period  and  not  the  total  expenditures  for  the  year.  Finally, 
PwC  explained  that  this  was  further  validated  by  testing  Carnegie  Mellon’s 
approval  of  the  invoices. 

Our  Response 

We  agree  that  the  phrase  "at  period-end"  was  used  throughout  the  documentation. 
However,  it  was  unclear  as  to  whether  this  meant  at  the  end  of  the  billing  period 
or  the  end  of  the  fiscal  year  because  the  testing  spreadsheet  identified  only  the 
invoice  amount,  the  total  expenditure  amount  at  the  end  of  the  fiscal  year,  and  a 
"Yes"  response  to  an  audit  step  stating,  "[ijnvoice  agrees  to  or  is  less  than  total 
expenditures  at  period-end.”  Regardless,  the  information  provided  did  not  change 
our  conclusions  because,  as  stated  in  our  finding,  the  testing  as  documented  did 
not  meet  the  audit  objective  for  testing  the  cash  management  requirement. 

PwC  Comments  on  Compliance  Testing  (second  paragraph) 

PwC  described  the  method  used  to  document  and  validate  its  understanding 
of  the  business  processes  and  internal  controls  over  cash  management.  PwC’s 
walkthrough  procedures  included  selecting  a  Federal  expenditure  transaction 
and  tracking  it  from  initiation  through  invoice  payment.  PwC  corroborated 
its  understanding  of  Carnegie  Mellon’s  processes  through  the  accounts  payable 
turnover  calculation  and  the  frequency  of  check  issuance.  PwC  stated  that  tracking 
the  expenses  to  purchase  orders  in  the  system  and  verifying  that  the  expenses 
were  charged  to  a  specific  award  would  suffice  for  testing  purposes. 

Our  Response 

We  appreciate  PwC’s  effort  to  clarify  its  reasoning  for  considering  that  audit 
documentation  taken  as  a  whole  was  sufficient  to  support  conclusions.  PwC  has 
also  agreed  to  perform  the  additional  testing  to  verify  that  expenditures  were 
paid  before  requesting  reimbursement.  During  our  follow-up  procedures  on  the 
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corrective  actions  taken,  we  will  verify  the  information  provided  in  PwC’s  response 
and  the  additional  testing  performed  to  determine  whether  there  is  sufficient 
overall  evidence  to  support  its  conclusions  on  the  cash  management  requirement. 

PwC  Comments  on  Testing  of  Internal  Controls  over  Compliance 
PwC  stated  that  it  documented  the  understanding  of  Carnegie  Mellon’s  internal 
control  using  the  Committee  of  Sponsoring  Organization  framework.  PwC 
indicated  that  it  performed  two  control  tests,  the  quarterly  letter  of  credit 
reconciliations  and  the  review  of  invoices.  The  reviewer  of  the  invoices  would 
check  whether  the  total  on  the  invoice  exceed  expenses  at  the  time  of  invoicing. 

Our  Response 

We  agree  that  PwC  documented  the  controls  using  the  Committee  of  Sponsoring 
Organization  framework  and  performed  two  control  tests.  However,  as  stated 
in  our  finding,  it  was  not  clear  in  the  audit  documentation  that  testing  whether 
the  invoice  exceeded  expenses  was  sufficient  to  meet  the  audit  objectives  of  the 
cash  management  requirement.  During  our  follow-up  procedures,  we  will  verify 
the  information  PwC  provided  in  its  response  to  determine  whether  the  internal 
control  identified  can  be  relied  on  to  meet  the  objectives  of  the  requirement. 

PwC  Comments  on  Audit  Documentation 

PwC  stated  that  due  to  an  intense  manual  process,  the  audit  documentation 
included  typographical  errors.  However,  PwC  believes  the  use  of  the  word 
"numerous"  overstates  the  magnitude  of  the  errors.  PwC  also  clarified  that  the 
25th  sample  item  reviewed  was  on  a  fixed-price  award  and  should  not  have  been 
chosen.  However,  PwC  does  not  believe  it  adversely  impacted  the  conclusions 
because  Carnegie  Mellon’s  internal  control  and  compliance  processes  are  the 
same  across  all  types  of  awards. 

Our  Response 

We  understand  human  error  and  the  audit  risk  associated  with  the  manual  entries 
in  the  audit  documentation.  We  do  not  consider  these  to  be  small  errors  because, 
as  documented,  the  auditors  appeared  to  select  some  transactions  from  a  prior 
audit  period  and  failed  to  identify  exceptions  in  the  testing  performed.  Because 
we  had  to  perform  retesting  to  verify  the  facts  for  the  transactions  being  tested 
and  almost  all  retested  sample  items  contained  errors,  we  do  not  consider  the 
magnitude  of  the  issue  to  be  overstated. 
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We  appreciate  the  clarification  that  the  25th  sample  item  was  on  a  fixed-price 
award.  PwC  had  previously  informed  us  that  this  sample  item  had  been 
erroneously  documented  and  that  a  different  item  had  been  reviewed. 

We  agree  with  PwC  that  this  item  should  not  have  been  chosen  because  the 
auditors  were  providing  an  opinion  on  compliance  with  Federal  program 
requirements  and  fixed-price  awards  are  not  part  of  major  Federal  programs 
for  the  OMB  Circular  A-133  audit.  Further,  PwC  planned  testing  for  a  sample 
of  25  items,  allowing  for  zero  exceptions,  to  support  conclusions.  Because  1  sample 
item  was  outside  the  scope  of  the  OMB  Circular  A-133  audit,  PwC  tested  only 
24  items.  As  a  result,  PwC  did  not  perform  the  testing  as  planned  to  support  its 
conclusions  on  compliance  with  the  cash  management  requirement.  However, 

PwC  has  agreed  to  perform  additional  cash  management  testing,  which  should 
resolve  this  issue. 
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November  13,  2015 


Randolph  R.  Stone 
Deputy  Inspector  General 
Policy  and  Oversight 

Inspector  General 
Department  of  Defense 
4800  Mark  Center  Drive 
Alexandria,  Virginia  22350 


Subject:  Quality  Control  Review  of  the  PricewaterhouseCoopers  LLP  FY  2014  Single  Audit  of  Carnegie 
Mellon  University  (Project  No.  D2015-DAPOSA-0171.000) 


Dear  Mr.  Stone: 

Thank  you  for  providing  a  draft  of  the  proposed  report  “Quality  control  Review  of  the 
PricewaterhouseCoopers  LLP  (“PwC”)  FY  2014  single  Audit  of  Carnegie  Mellon  University”  (the  “Draft 
Report”).  We  have  reviewed  the  Draft  Report  and  we  appreciate  the  opportunity  to  provide  additional 
information  and  clarification  on  the  potential  findings  noted  therein. 

The  Department  of  Defense  Office  of  Inspector  General  (“OIG”)  indicated  that  Carnegie  Mellon  University 
(“CMU”)  complied  with  OMB  Circular  A-133  reporting  requirements.  However,  the  OIG  also  indicated 
that  the  PwC  did  not  fully  comply  with  auditing  standards  and  OMB  Circular  A-133  requirements  for  the 
CMU  FY  2014  single  audit.  Specifically  the  OIG  indicates  that  PwC  did  not  adequately  perform  and 
document  audit  procedures  for  our  review  of  the  cash  management  requirements.  Therefore  the  OIG  has 
concluded  we  need  to  complete  additional  work  to  support  our  audit  conclusions  and  overall  opinion  on 
compliance  with  the  cash  management  requirement  procedures  for  the  research  and  development  (‘R&D’) 
cluster. 

The  OIG  recommends  that  PwC  document  and  perform,  as  necessary,  sufficient  audit  procedures  to 
support  conclusions  on  CMU’s  internal  control  over  compliance  with  cash  management  requirements.  In 
addition  it  is  recommended  that  PwC  provide  to  the  OIG  documentation  to  support  that  PwC  has  taken 
corrective  action  to  address  the  deficiencies  reported  in  the  Draft  Report. 

The  Draft  Report  describes  what  the  OIG  considers  to  be  two  deficiencies.  First  that  the  procedures  PwC 
did  perform  did  not  go  far  enough  to  adequately  demonstrate  that  CMU  pays  R&D  program  costs  with 
CMU  funds  before  requesting  reimbursement  from  the  Federal  Government.  Second,  one  of  the  PwC 
testing  spreadsheets  contained  typographical  errors. 

PwC  will  perform  the  additional  cash  management  test  requested  for  fiscal  year  2014.  We  will  select  an 
appropriate  sample  of  invoices  and  compare  the  date  that  the  invoice  was  actually  paid  to  the  date  CMU 
requested  reimbursement  for  the  invoices  chosen.  In  addition  we  will  correct  the  information  errors  OIG 
found  on  one  of  our  test  spreadsheets. 
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While  we  believe  that  taken  as  a  whole,  our  documented  procedures  over  compliance  and  internal  control 
over  compliance  did  adequately  support  our  opinion  over  compliance  for  the  R&D  cluster,  we  respect  the 
OIG  point  of  view  and  accordingly  are  taking  the  requested  corrective  action.  In  addition  while  we  agree 
that  one  of  our  spreadsheets  contained  some  inaccurate  pieces  of  information,  we  do  not  believe  that  those 
errors  negatively  impacted  the  audit  conclusions  we  reached. 

We  have  included  in  Appendix  A  a  more  detailed  response  to  the  specific  facts  and  circumstances  noted  in 
the  Draft  Report.  If  after  reading  our  response,  you  would  like  to  see  any  of  the  procedures  we  refer  to, 
such  as  our  internal  control  documentation,  we  would  be  able  to  send  to  copies  of  the  procedures  we 
performed  and  the  related  results  of  our  testing. 

PwC  welcomes  the  opportunity  to  discuss  with  the  OIG  in  the  future  the  level  of  audit  testing  required  by 
the  existing  and  applicable  guidance  to  allow  an  auditor  to  conclude  on  the  objective  of  ensuring  that 
recipients  of  Federal  awards  are  paying  invoices  with  recipient  funds  before  requesting  a  drawdown  of 
Federal  Funds  for  reimbursement  of  those  recipient  funds. 

Thank  you  for  your  consideration  of  our  response  to  the  Draft  Report.  We  would  be  pleased  to  discuss  this 
with  you  at  your  convenience.  Please  call  Riva  Mirvis  at  703-918-3886  to  arrange  a  mutually  convenient 
time. 


Sincerely, 


Riva  Mirvis 
Engagement  Partner 
PricewaterhouseCoopers 
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Appendix  A 

Compliance  Testing  Section 
[OIG  comments] 

PwC  selected  a  sample  of  reimbursement  requests  and  performed  two  audit  procedures  to  test  Carnegie 
Mellon’s  compliance  with  the  cash  management  requirement.  PwC  verified  that  the  amount  requested 
for  reimbursement  did  not  exceed  total  expenditures  incurred  for  the  award.  The  audit  documentation 
was  not  clear  on  whether  the  auditor  used  cumulative  incurred  expenses  at  year  end  or  expenses 
incurred  at  the  time  of  the  reimbursement  request.  In  either  case ,  this  audit  procedure  did  not  provide 
evidence  that  met  the  audit  objective  for  the  requirement  because  PwC  is  relying  on  the  expense 
accounts2  without  documenting  evidence  of  Carnegie  Mellon’s  payment  of  the  expenses.  PwC  also 
verified  that  the  amount  received  matched  the  amount  requested;  however,  this  procedure  also  did  not 
achieve  the  audit  objective  for  the  requirement. 

[PwC’s  response] 

In  our  dual  purpose  control/compliance  testing,  we  clearly  defined  our  testing  objectives  to  ensure  that  the 
invoice  amount  agrees  to  or  is  less  than  the  total  expenditures  at  period-end.  This  is  documented  in  3 
places:  “Test  of  Compliance  template”  step  3,  Procedures  tab  item  2-3)  in  our  Test  of  Compliance 
workpaper  and  Column  H  in  our  Test  of  Compliance  workpaper.  In  our  Test  of  Compliance  workpaper  - 
Column  H  is  labelled  as  ‘Invoice  agrees  to  or  is  less  than  total  expenditures  at  period-end’ .  We  ensured 
and  documented  for  each  sample  that  the  invoiced  amount  did  not  exceed  the  total  expenditures  at 
period-end.  We  did  not  use  Column  G  as  Column  G  is  labelled  as  Total  Expenditures  while  Column  H  is 
expenditures  at  period  end.  This  is  further  validated  when  we  performed  control  testing  for  key  control  2 
-  review  of  invoices.  These  workpapers  were  provided  while  OIG  was  on  site,  however,  in  case  you  would 
like  to  see  these  workpapers,  we  will  send  a  copy  to  you  under  a  separate  cover. 

In  summary,  for  dual  purpose  control  and  compliance  testing,  we  selected  25  samples  and  performed  the 
following  procedures. 

a.  For  each  draw  down  or  invoice,  we  ensured  that  it  was  reviewed  and  that  total  draw  or  amount 
invoiced  did  not  exceed  expenses  at  the  time  of  invoicing.  Column  H  of  the  testing  template  states 
"Invoice  agrees  to  or  is  less  than  total  expenditures  at  period-end"  and  this  test  was  for  the  billing 
period  and  not  corresponding  to  Column  G  of  total  expenditures  for  the  year. 

b.  Key  control  2  -  review  of  invoices  -  were  tested  for  selected  samples.  We  ensured  that  invoices 
were  properly  approved  by  responsible  management.  The  review  of  invoices  is  subjected  to 
extensive  procedures  by  CMU.  We  tested  that  reviewers  ensured  that  the  drawdowns  or  invoiced 
amounts  did  not  exceed  expenditures  at  the  time  as  documented  in  Column  P. 

[OIG  comments] 

We  asked  PwC  to  explain  why  the  documentation  did  not  include  evidence  of  the  date  Carnegie  Mellon 
paid  expenses.  PwC  informed  us  that  their  reliance  on  expense  accounts  is  based  on  the  understanding 
that  Carnegie  Mellon  issues  monthly  reimbursement  requests  for  the  previous  month’s  expenses  and  an 
analysis  of  the  annual  accounts  payable  turnover  ratio.  The  turnover  ratio  used  is  a  broad  assessment  of 
the  annualized  average  time  it  takes  Carnegie  Mellon  to  pay  the  expenses,  but  does  not  provide  proper 
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evidence  of  actual  payment  before  the  request  for  reimbursement.  Additionally,  PwC  did  not  document 
audit  procedures  to  validate  their  understanding  of  Carnegie  Mellon’s  processes.  As  a  result,  the  audit 
documentation  did  not  include  sufficient  evidence  that  the  auditors  met  the  audit  objective  to  conclude  on 
compliance  with  the  cash  management  requirement. 

[PwC’s  response] 

We  documented  and  validated  our  understanding  of  internal  control  over  various  business  processes 
including  the  payables  and  expense  cycles,  the  payroll  cycle,  sponsored  receivables  cycle,  sponsored 
revenue  cycle  and  the  treasury  cycle  from  start  to  finish  through  walkthrough  procedures.  In  addition  we 
obtained  an  understanding  of  internal  controls  surrounding  cash  management  and  documented  our 
understanding.  Our  walkthrough  procedures  included  selecting  a  federal  expenditure  transaction  and  we 
followed  it  through  from  initiation  of  transaction  until  the  invoice  was  paid.  During  the  walkthrough,  we 
obtained  system  screenshots,  a  proof  of  approvals,  assignment  to  a  particular  award,  3-way  match  with 
PO,  invoice  and  a  receipt,  and  payment  processing.  We  corroborated  our  understanding  of  CMU’s  process 
through  the  accounts  payable  turnover  calculation  and  frequency  of  check  issuance.  As  validated  through 
the  walk  through,  expenses  are  billed  in  the  month  following  the  date  of  the  expenditure,  not  prior. 
Invoices  to  the  Federal  Government  or  sponsor  include  various  information  such  as  period  covered, 
invoice  date,  net  cumulative  invoiced  amount,  current  amount  billed  and  management  certification. 
Additionally,  a  significant  portion  of  direct  costs  (approximately  76%)  are  compensation  for  personnel 
services,  stipend  and  allowances.  Through  our  direct  cost  testing  for  compensation,  we  traced  46  samples 
to  cash  payments  to  ensure  they  were  paid  timely  and  noted  no  exceptions.  Through  other  direct  costs 
testing  (equipment,  consultant,  travel,  and  other  expenses),  we  traced  11  samples  to  POs  or  invoices  in  the 
accounts  payable  systems  and  it  validates  that  they  were  processed  through  normal  accounts  payable 
system.  This  testing  combined  with  ensuring  CMU  has  an  appropriate  process  to  this  end  and  analytical 
documentation,  we  believe  our  cash  management  work  adequately  meets  this  audit  objective. 

In  addition,  through  our  understanding  of  the  internal  control  environment,  we  have  learned  the 
following: 

a.  The  majority  of  CMU’s  awards  are  reimbursement  basis  and  they  have  one  LOC  with  NIH. 
Other  federal  awards  are  directly  billed  (invoiced)  for  reimbursement  to  sponsoring  federal 
government  agencies  or  the  prime  award  recipients. 

b.  Our  walkthrough  confirmed  that  all  procurements  from  vendors  require  purchase  orders. 
Purchase  orders  go  through  approval  workflows  within  the  University  and  when  purchase 
orders  are  approved,  such  POs  are  linked  to  particular  awards. 

c.  Once  vendor  invoices  are  received  and  matched  with  POs  and  the  receipt  of  goods/services, 
transactions  are  sent  to  batch  processing  for  payments  daily  and  checks/wire  payments  are 
issued  almost  daily.  At  June  30,  2014  and  2013,  outstanding  checks,  including  federal  and 
non-federal  expenditures  were  approximately  $6.4  million  for  both  2014  and  2013  year- 
ends  and  not  material  to  R&D  and  for  schedule  of  expenditures  for  federal  awards  (‘SEFA’). 

d.  LOCs,  AR  and  cash  on  hand  are  reconciled  quarterly  basis  and  reviewed  by  appropriate 
management  (cash  management  key  control  1).  LOC  drawdown  is  prepared  monthly  to 
capture  previous  month’s  expenses.  During  preparation  and  review  of  LOC,  cumulative 
expenses,  cumulative  drawdowns  and  current  period’s  expenses  are  downloaded  from 
Oracle  and  reviewers  ensure  next  drawdowns  do  no  exceed  cumulative  expenses  at  the  time. 

e.  Invoices  to  request  reimbursements  are  prepared  monthly  in  the  following  month  (e.g.  July 
expenses  is  billed  in  August.)  All  invoices  to  sponsoring  government  agencies  and  prime 
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recipients  are  reviewed  by  appropriate  management  (cash  management  key  control  2). 
Invoices  include  key  information  such  as  expense  period,  invoice  date,  award  information, 
cumulative  cost,  current  amount  requested  for  reimbursement  and  certification  of  the 
management.  During  the  invoice  review,  reviewer  ensure  mathematical  accuracy  of 
invoices,  costs  requested  for  reimbursements  are  for  prior  period  and  not  estimated 
amount,  all  payments  requested  are  for  appropriate  purpose  and  in  accordance  with  the 
agreement  set  forth  in  the  application  and  award  document.  We  believe  that  this  control  is  a 
key  for  cash  management  to  ensure  reimbursement  requests  were  prepared  properly. 

Based  on  our  understanding,  we  assessed  overall  risk  related  to  cash  management  to  be  low  and  planned 
the  testing  of  internal  control.  Two  key  controls  were  identified  as  noted  above. 

In  addition,  to  test  compliance  requirements  for  cash  management,  we  have  calculated  accounts  payable 
turn  over  to  be  21  days  and  considering  there  wasn’t  material  long  outstanding  checks,  we  determined  that 
‘paid’  for  cash  management  compliance  purpose  means  expenses  are  processed  in  normal  course  of 
business.  By  tracing  expenses  to  POs  in  AP  system  to  ensure  they  were  directly  charged  to  specific  award 
will  suffice  for  the  purpose  of  the  testing. 

Testing  of  Internal  Controls  over  Compliance 

[OIG  comments] 

PwC  performed  two  audit  procedures  for  testing  internal  controls.  PwC  reviewed  Carnegie  Mellon’s 
approval  of  the  reimbursement  request  and  Carnegie  Mellon’s  letter  of  credit  quarterly  reconciliations. 
The  audit  documentation  did  not  provide  clear  evidence  of  how  the  individual  approving  the 
reimbursement  ensured  compliance  with  the  cash  management  requirement.  The  test  of  quarterly 
reconciliations  provides  limited  coverage  for  the  overall  cash  management  requirement  because  the 
letter  of  credit  is  applicable  to  only  5.7  percent  ofFY  2014  Federal  expenditures.  As  a  result,  we 
determined  that  the  audit  documentation  did  not  provide  sufficient  evidence  to  support  the  audit 
conclusions  on  internal  control  and  to  plan  the  compliance  testing  for  the  cash  management 
requirement. 

[PwC  response] 

We  have  documented  our  understanding  of  CMU’s  internal  control  environment  surrounding  the  cash 
management  requirements  included  in  the  OMB  Compliance  supplement  using  a  COSO  format  template. 
Risk  assessment  for  cash  management  was  performed  as  part  of  our  engagement  planning  and  gaining  an 
understanding  of  internal  control  procedures.  Risk  related  to  internal  control  over  cash  management 
compliance  was  determined  and  documented  to  be  low.  We  have  identified  2  key  controls  over  the  cash 
management  compliance  requirements.  As  OIG  described  in  the  Draft  Report,  the  first  key  control  (CMi) 
was  the  test  of  quarterly  LOC  reconciliations.  The  second  key  control  (CM2)  was  review  of  invoices. 

For  Key  Control  2  (CM2),  we  have  performed  the  following  procedures  for  25  samples.  (7  drawdowns  and 
18  invoices).  This  was  a  dual  purpose  test. 

a.  For  each  draw  down  or  invoice,  ensure  that  it  was  reviewed  that  total  draw  or  amount  invoiced  did 
not  exceed  expenses  at  the  time  of  invoicing.  Column  H  of  the  testing  worksheet  states  “Invoice 
agrees  to  or  is  less  than  total  expenditures  at  period-end”  is  for  the  monthly  billing  period  and  not 
corresponding  to  Column  G  of  total  expenditures  for  the  year. 

b.  To  ensure  invoices  were  properly  approved  by  responsible  management. 
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Audit  Documentation 
[OIG  comments] 

We  noted  that  the  auditor’s  spreadsheet  documenting  the  internal  control  and  compliance  testing 
identified  reimbursement  request  dates  that  were  for  a  prior  fiscal  year  and  payment  received  dates  that 
were  before  the  reimbursement  request  date.  Due  to  the  discrepancies  identified,  we  decided  to  perform 
retesting  and  selected  6  of  the  25  sample  items  for  review.  We  were  able  to  determine  that  the  dates  on 
the  spreadsheet  were  entered  incorrectly  for  five  of  the  six  sample  items.  However,  Carnegie  Mellon 
informed  us  that  the  sixth  item  was  for  a  fixed-price  award,  which  would  not  be  subject  to  OMB  Circular 
A-133  audit  requirements.  When  asked,  PwC  explained  that  the  sample  item  described  on  the 
spreadsheet  was  not  the  reimbursement  request  actually  tested.  During  our  retesting,  we  noted  errors  in 
the  audit  documentation  including  the  identification  of  a  different  approver  for  the  reimbursement 
request  than  the  one  identified  on  the  testing  spreadsheet  and  errors  in  dates.  Due  to  these  inaccuracies, 
we  are  unable  to  rely  on  the  audit  documentation  for  the  cash  management  requirement. 

[PwC’s  response] 

During  the  exit  conference  held  on  September  3,  2015,  we  were  told  that  the  documentation  of  our  cash 
management  testing  was  not  fully  accurate  and  that  there  were  some  wrong  dates,  wrong  amounts,  and  a 
wrong  approver.  In  addition,  one  sample  was  entirely  the  wrong  sample  tested. 

We  agree  that  they  included  typographical  errors  due  to  the  intense  manual  process.  Oracle  invoicing  may 
not  always  generate  invoices  in  accordance  with  costs  incurred  due  to  CMU’s  task  funding  set  up  in  Oracle. 
Actual  costs  incurred  were  used  to  determine  invoice  amounts  that  were  submitted  to  the  sponsor,  which 
agree  to  Oracle  generated  accounting  reports.  During  our  audit  process,  we  were  provided  with  the  list  of 
Oracle  invoices,  and  then  match  actual  invoices  that  were  sent  to  the  sponsoring  agencies,  and  identify 
expenditures  associated  with  the  actual  invoice.  All  data  were  pulled  from  the  hard  copy  information 
provided  by  CMU  and  manually  typed  in  our  workpaper.  We  also  note  that  the  OIG  was  able  to  re-perform 
at  CMU  our  cash  management  testing  and  reach  the  same  conclusion  for  CMU’s  cash  management 
compliance  requirements  as  we  did,  even  though  our  documentation  did  include  a  small  number  of  errors. 
We  believe  that  the  use  of  the  word  ‘numerous’  to  describe  what  was  found  overstates  the  magnitude  of  the 
errors. 

Regarding  our  25th  sample  transaction  chosen,  we  agree  it  was  for  a  fixed-price  award,  although  this 
award  is  not  subject  to  inclusion  in  the  CMU  SEFA  and  therefore  should  not  have  been  chosen,  we  do  note 
that,  the  internal  control  and  compliance  processes  CMU  has  in  place  are  the  same  across  all  types  of 
awards.  Therefore  we  do  not  believe  that  use  of  the  fixed  price  transaction  in  any  way  adversely  impacts 
the  conclusions  we  drew  from  our  sample 
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Acronyms  and  Abbreviations 


CIGIE  Council  of  Inspectors  General  on  Integrity  and  Efficiency 
OMB  Office  of  Management  and  Budget 
PwC  PricewaterhouseCoopers  LLP 
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Whistleblower  Protection 
U.S.  Department  of  Defense 


The  Whistleblower  Protection  Enhancement  Act  of  2012  requires 
the  Inspector  General  to  designate  a  Whistleblower  Protection 
Ombudsman  to  educate  agency  employees  about  prohibitions 
on  retaliation,  and  rights  and  remedies  against  retaliation  for 
protected  disclosures.  The  designated  ombudsman  is  the  DoD  Hotline 
Director.  For  more  information  on  your  rights  and  remedies  against 
retaliation,  visit  www.dodig.mil/programs/whistleblower. 


For  more  information  about  DoD  IG 
reports  or  activities,  please  contact  us: 

Congressional  Liaison 

congressional@dodig.mil;  703.604.8324 

Media  Contact 

public.affairs@dodig.mil;  703.604.8324 

For  Report  Notifications 

http://www.dodig.mil/pubs/email_update.cfm 

Twitter 

twitter.com/DoD_IG 


DoD  Hotline 

dodig.mil/hotline 


DEPARTMENT  OF  DEFENSE  |  INSPECTOR 

4800  Mark  Center  Drive 
Alexandria,  VA  22350-1500 
www.dodig.mil 

Defense  Hotline  1.800.424.9098 


